Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: Adaptive closed-loop stimulation (ACLS) works by iterating through three steps. (A) First, the system observes the evoked response to a specific pattern of stimulation. Here, we record multi-unit activity, but this could be any measurable variable (e.g. bold signal, local field potential, single units, or behavior). (B) Second, the ACLS evaluates an error function that determines the difference between the evoked response and the desired response. Here, we use Euclidean distance to a target pattern of neural activity, but this can be any error function (e.g. cosine similarity, correlation) to a target neural or behavioral state. (C) Third, a machine learning algorithm updates the stimulation pattern in order to reduce the error function. By iteratively progressing through these steps, the algorithm learns a stimulation pattern that minimizes the error function and, thus, produces a desired neural or behavioral response. ACLS pseudocode is supplied in .

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Activity Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Schematic of experiment. A 5-layer deep convolutional neural network (CNN) was trained to classify 10 numeric digits from the MNIST database (see Methods). Images were downsampled to a resolution of 9×9 pixels before training. Numerical images were delivered as ‘stimulation’ to the trained network and activity in the last hidden layer was considered the ‘response’. For each response, an error function (Euclidean distance) between the evoked and target response was computed (yellow box). This error was provided to the ACLS which used a stochastic learning algorithm (see panel E and Methods) to iteratively improve the stimulation pattern (purple box). (B) The target response of the CNN, taken as the response to a randomly selected image of the number 1. (C) Distribution of explained variance captured by the principal components (PCs) of the activity in the response layer of the CNN. (D) Response of the neurons in the response layer to 250 different images of the numbers 0, 1, 2 and 3, colored in green, gray, purple and blue, respectively. Responses are projected into the two-dimensional subspace created by the first two PCs. The orange star shows the target response (from B). (E) Schematic of ACLS stochastic learning algorithm. During a block, the algorithm generates a set of new patterns (column of dots) by perturbing the current ‘best’ pattern (black dot in previous column). Of these, the pattern that minimizes the error function of the system (y-axis) becomes the new ‘best’ pattern. This process repeats during each block, keeping the previous best stimulation pattern if none of the new patterns decreased error (red-circle and dotted lines). (F) Example learning trajectory of ACLS. Orange star denotes the target response. Orange line shows the trajectory of responses as the ACLS system learns. Each point denotes the mean response from a single block, with the shaded region around that point indicating the SEM in response (each block included 5 repetitions of 10 unique stimulation patterns). Initial orange point was mean response on block 1, with dotted black line indicating its displacement from the initial condition. In this simulation, Gaussian noise (µ = 0, σ = 0.5) was added to the response of the fully connected layer on each trial. Inset shows region indicated by gray box in full plot, with distributions of the ACLS learning path removed for clarity. (G) Annealing factor (λ) controls the magnitude of perturbations when generating new stimulation patterns. To arrive at global minima, while avoiding local minima, the stochastic learning algorithm increases/decreases the magnitude of the random perturbations to the current best pattern depending on whether a new best stimulus was/was not found during the previous block (see Methods and ). (H-I) Error (ϵ) between ACLS-evoked responses and target response across blocks, computed in both the (H) 2D PC space and (I) full dimensional space. Solid line shows mean error per block (N=50 stimulations per block, x-axis). For comparison, the dotted line shows the error over learning when ACLS minimized error in the full-dimensional space (N=50 stimulations per block). (J) Stimulus classification accuracy by the CNN during learning. Orange line shows the fraction of ACLS-generated stimuli that were classified as the target category (e.g. the number 1). Black line shows the fraction of stimuli classified as the number 0 (the initial stimulation pattern provided to the ACLS; dotted line shows the classification rate of this initial image prior to the first block).

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Activity Assay, Blocking Assay, Comparison, Generated

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) ACLS was tasked with controlling different hidden layers of a CNN (colored boxes) by reducing error in either the full dimensional or reduced dimensional space. Display follows . (B-G) Example evolution of learned stimulation patterns when calculating error in (B-D) full or (E-G) reduced dimensional space. Errors were normalized to distance between initial point and target points in full or reduced dimensional space. Numbered images show the best learned stimulation pattern during a subset of 100 blocks (each block consisted of 1 repetition of 20 unique stimulation patterns). For these simulations, no noise was added to responses of fully connected layer. “Initial” shows the initial stimulation pattern provided to ACLS. “Target” shows the pattern originally used to generate the target response in the network. As seen in B-C, the patterned learned by ACLS was similar to the original pattern when controlling the earlier layers. For display purposes, the image intensity was normalized to the minimum and maximum for each example run (i.e. within a row).

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Blocking Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Traces showing the impact of different noise magnitudes on the repeated presentation of the same stimulus pattern. On each trial, white Gaussian noise (µ = 0, σ = 0, 0.5, 1, 3, or 5) was added to the response of the final hidden layer (equates to an average SNR of ∼18, 9, 3, and 2, respectively). Traces show the block-averaged error of evoked response to the same stimulation pattern across blocks (e.g. the initial pattern given to ACLS; N=2 repetitions per block). N=50 replications for each noise level. (B) Noise in response impaired ACLS learning. Noise levels match those in A. Repeating stimuli mitigated the impact of noise: light, medium, and dark gray lines show the error across blocks with 1, 5, and 10 repetitions of each stimulus when noise was held constant at σ = 3. Lines show the median block-averaged error of evoked response, normalized to the average error of the first block (N=50 runs for each noise level). Shaded regions show inter-quartile range (IQR). (C) As in A, but shows how drift in the weights of the fully connected layer affects the response to the repeated presentation of the same stimulus pattern. On each trial, white Gaussian noise was accumulatively added to the weights of the fully connected layer (σ of noise distribution was 0, 0.93, 9.3, or 93% of the initial standard deviation in weights). N=50 replications for each drift level. (D) As in B, but shows the ACLS can compensate for most levels of drift. Colors match drift levels in C. (E) Bar plot showing that increasing the number of image classes to be classified by the CNN increased the dimensionality of the final hidden layer. Dimensionality was measured as the number of principal components needed capture ≥9 0% of the variance. (F) As in B and D, but shows increasing the complexity of the representational space slows ACLS learning, but does not prevent learning. Colors match image classes in E.

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Blocking Assay, Standard Deviation

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Example trajectory of ACLS learning to control the final hidden layer of CNN using a cosine similarity error function. The error function was ε = 1 − cos θ where and T and V were the target and current response vectors, respectively. Display follows . (B) ACLS decreased the cosine similarity error function in both full (blue) and two-dimensional PC space (red). Lines and shaded regions show the median and IQR of 50 replications. (C) Varying the number of stimulation patterns per block impacted ACLS learning. Learning was quantified when using 3 (green), 6 (purple), or 10 (orange) unique stimulation parameters per block. Solid lines show median error of evoked responses over learning with shaded regions showing IQR over N=50 replications. Higher number of stimulation patterns per block resulted in more reliable learning (demonstrated by a lower variance in the learning curve), but required more total stimulations, shown as an increase in the cumulative number of trials across blocks (dashed lines, right y-axis).

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Control, Blocking Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Example multi-unit activity in response to random electrical stimulation in the primary visual cortex of an anesthetized mouse (see Supplemental Methods for details). Display follows . (B) The electrical stimulation patterns that evoked the corresponding colored responses in A. (C) Example trajectory of ACLS learning to reproduce arbitrarily-defined population responses (previously elicited by random electrical stimulation). Gray dots show the responses to random electrical stimulation projected into a 2D space defined by the first two principal components (as in ). Two responses were selected as target responses for ACLS to achieve (orange and purple stars; corresponding to firing and stimulation patterns shown in A-B). ACLS was tasked with learning to reproduce these responses, starting from the stimulation pattern that produced the response of the opposite target (e.g. purple run starts at orange and learns towards purple, and vice versa). Panels in C show the learning trajectory of the algorithm across 22 blocks. Display follows 4C. (D) Corrected error over time. Display follows and .

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Activity Assay, Produced

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Experiment schematic. Awake, head-fixed mice were presented with visual stimuli while recording multi-unit neural activity in the primary visual cortex (V1). Visual stimuli were 200ms in duration, interleaved with 1000ms presentations of a gray screen. Black trace shows example raw trace of visual stimulus-evoked multi-unit spiking activity (red dots). (B) Example multi-unit activity response to the purple (top) and orange (bottom) outlined images in A. (C) Distribution of responses to 10 different visual stimuli, each presented 20 times (gray dots indicate response to one stimulus). Purple and orange stars show the target response to the purple and orange-outlined stimuli. Shaded regions denote STD of 20 visual stimulus presentations of each stimulus. (D) Explained variance in multi-unit activity captured by the principal components. (E) Schematic of ACLS learning to reproduce visual stimulus-evoked neural responses. Electrical stimulation was applied with a gray screen to match baseline visual input. An example stimulation pattern across 7 stimulation sites is shown in purple. Black trace shows example raw trace of stimulation followed by multi-unit spiking activity (red dots). (F) Example timecourse of learning. Orange and purple stars indicate the two target responses (as in B and C, dotted lines around each star shows STD of visual response). Targets were learned by two simultaneous, but independent, runs of the ACLS algorithm. Orange and purple lines show the 2D trajectory of learning over blocks. Shaded region around each step indicates the SEM of responses in that block (N=12 blocks, each with 5 repetitions of 6 unique stimulation patterns). Individual purple and orange points show response to repetitions of the initial stimulus pattern (1 repetition per block). Transparency of points scales with time (light → dark). The movement of these points show the system drifts over time. Black dotted line denotes borders of 2D visual response space shown in C. Inset shows region denoted by gray box in full sized image. Note that the only input to the ACLS algorithm was the initial stimulation pattern (purple pattern in E) and the error between each evoked response and the target response (purple and orange lines). (G-H) Error between ACLS-evoked responses and target response across blocks in (G) 2D response space and (H) full-dimensional space. Solid orange and purple lines show mean error per block (Euclidean distance; N=30 stimulations); shaded regions show SEM per block. Dotted black line denotes noise floor, quantified as the average pairwise Euclidean distance between repetitions of the same stimulation pattern (N=5 repetitions of 6 unique stimulation patterns per block). Dashed purple and orange lines show the drift in the response of the initial stimulation, measured as the distance to the target response. Lines for initial stimulus repetitions and noise lines reflect sliding window mean of 6 blocks. (I- Error over time, corrected for drift of initial stimulation response over blocks (see Methods). As in G-H, solid lines show error of learned responses across blocks. Dashed lines show the corresponding noise floors (also corrected).

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Activity Assay, Blocking Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Multi-unit responses during ACLS learning for the example session shown in . Each plot shows the best response during a given block (e.g. the response with the lowest error). Orange and purple denote responses for the two simultaneous ACLS runs. Note: the y-axes are independently scaled in order to show the full dynamic range of response values. (B) Target responses for each run. Colors match panels A and C (C) Stimulation pattern that evoked the corresponding responses in A (aligned by column). (D) Learning trajectory in PC space associated with A-C. Numbers correspond to example blocks in A and C.

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Blocking Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A-G) The left panel of each pair of panels shows the (A, C, E, G) 2D trajectory of example ACLS runs in awake mice and the (B, D, F, H) associated drift-corrected error across blocks. The panels follow the displays in and for the left and right panels, respectively.

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques:

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Corrected error over time for all ACLS runs (N = 48) in all recording sessions (N=24), across all mice (N=16). Display follows . Individual orange and gray lines show error over time of individual successful and failed ACLS runs, respectively. Runs were considered successful if the mean error during the final block was less than the first block. Red line shows mean trajectory across successful runs. For visualization, the x-axis truncated at 15 blocks (N=6 runs were longer than 15 blocks, all continued to decrease in error). (B) Pairwise comparisons of drift-corrected error (orange dots) and noise (black dots) at blocks 1, 5 and 10 across all ACLS runs (N=48). Δ N is the mean difference between the error and the noise of each run (i.e. gray lines). (C) Table of success rate of the ACLS runs across multiple sessions, quantified by a reduction in error over learning (left column), a negative exponential fit to the learning curve (middle column), and specific learning towards one target (relative to the other; right column). See text and Methods for details of the statistics.

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: Blocking Assay

Journal: bioRxiv

Article Title: Learning to Control the Brain through Adaptive Closed-Loop Patterned Stimulation

doi: 10.1101/2020.03.14.992198

Figure Lengend Snippet: (A) Corrected error over time across N=34 runs in N=68 recordings in N=10 anesthetized mice. Display follows 5A. Figure cropped to 15 blocks to allow for visualization of all ACLS runs. N=5 continued to decrease their error after this point. (B) Pairwise comparisons of error and noise levels at three time points. Display follows .

Article Snippet: The Adaptive Closed-Loop Stimulation (ACLS) system was implemented in a custom-built MATLAB software package and GUI.

Techniques: